Postoperative corneal topography generation based on attention mechanism and Pix2Pix network / 国际眼科杂志(Guoji Yanke Zazhi)

AIM:To explore the use of attention mechanism and Pix2Pix generative adversarial network to predict the postoperative corneal topography of age-related cataract patients undergone femtosecond laser arcuate keratotomy.METHODS:In this retrospective case series study, the 210 preoperative and postoperative corneal topographies from 87 age-related cataract patients(105 eyes)undergoing femtosecond laser arcuate keratotomy at Shanxi Eye Hospital between March 2018 and March 2020 were selected and divided into a training set(180)and a test set(30)for model training and testing. The peak signal-to-noise ratio(PSNR), structural similarity(SSIM)and Alpins astigmatism vector analysis were used to compare the accuracy of postoperative corneal topography prediction under different attention mechanisms.RESULTS:The model based on attention mechanism and Pix2Pix network can predict postoperative corneal topography, among which the model based on Self-Attention mechanism has the best prediction effect, with PSNR and SSIM reaching 16.048 and 0.7661, respectively. There were no statistically significant differences in the difference vector, difference vector axis position, surgically induced astigmatism, and correction index between real and generated corneal topography on the 3mm and 5mm rings(all P>0.05).CONCLUSION:Based on the Self-Attention mechanism and Pix2Pix network, the postoperative corneal topography can be well predicted, which can provide reference for the surgical planning and postoperative effects of ophthalmic clinicians.

Construction of an epileptic seizure prediction model using a semi-supervised method of generative adversarial and long short term memory network combined with Stockwell transform / 南方医科大学学报

OBJECTIVE@#To propose a semi-supervised epileptic seizure prediction model (ST-WGAN-GP-Bi-LSTM) to enhance the prediction performance by improving time-frequency analysis of electroencephalogram (EEG) signals, enhancing the stability of the unsupervised feature learning model and improving the design of back-end classifier.@*METHODS@#Stockwell transform (ST) of the epileptic EEG signals was performed to locate the time-frequency information by adaptive adjustment of the resolution and retaining the absolute phase to obtain the time-frequency inputs. When there was no overlap between the generated data distribution and the real EEG data distribution, to avoid failure of feature learning due to a constant JS divergence, Wasserstein GAN was used as a feature learning model, and the cost function based on EM distance and gradient penalty strategy was adopted to constrain the unsupervised training process to allow the generation of a high-order feature extractor. A temporal prediction model was finally constructed based on a bi-directional long short term memory network (Bi-LSTM), and the classification performance was improved by obtaining the temporal correlation between high-order time-frequency features. The CHB-MIT scalp EEG dataset was used to validate the proposed patient-specific seizure prediction method.@*RESULTS@#The AUC, sensitivity, and specificity of the proposed method reached 90.40%, 83.62%, and 86.69%, respectively. Compared with the existing semi-supervised methods, the propose method improved the original performance by 17.77%, 15.41%, and 53.66%. The performance of this method was comparable to that of a supervised prediction model based on CNN.@*CONCLUSION@#The utilization of ST, WGAN-GP, and Bi-LSTM effectively improves the prediction performance of the semi-supervised deep learning model, which can be used for optimization of unsupervised feature extraction in epileptic seizure prediction.

Research progress on medical image dataset expansion methods / 生物医学工程学杂志

Computer-aided diagnosis (CAD) systems play a very important role in modern medical diagnosis and treatment systems, but their performance is limited by training samples. However, the training samples are affected by factors such as imaging cost, labeling cost and involving patient privacy, resulting in insufficient diversity of training images and difficulty in data obtaining. Therefore, how to efficiently and cost-effectively augment existing medical image datasets has become a research hotspot. In this paper, the research progress on medical image dataset expansion methods is reviewed based on relevant literatures at home and abroad. First, the expansion methods based on geometric transformation and generative adversarial networks are compared and analyzed, and then improvement of the augmentation methods based on generative adversarial networks are emphasized. Finally, some urgent problems in the field of medical image dataset expansion are discussed and the future development trend is prospected.

Application of generative adversarial network in magnetic resonance image reconstruction / 生物医学工程学杂志

Magnetic resonance imaging (MRI) is an important medical imaging method, whose major limitation is its long scan time due to the imaging mechanism, increasing patients' cost and waiting time for the examination. Currently, parallel imaging (PI) and compress sensing (CS) together with other reconstruction technologies have been proposed to accelerate image acquisition. However, the image quality of PI and CS depends on the image reconstruction algorithms, which is far from satisfying in respect to both the image quality and the reconstruction speed. In recent years, image reconstruction based on generative adversarial network (GAN) has become a research hotspot in the field of magnetic resonance imaging because of its excellent performance. In this review, we summarized the recent development of application of GAN in MRI reconstruction in both single- and multi-modality acceleration, hoping to provide a useful reference for interested researchers. In addition, we analyzed the characteristics and limitations of existing technologies and forecasted some development trends in this field.

An image classification method for arrhythmias based on Gramian angular summation field and improved Inception-ResNet-v2 / 生物医学工程学杂志

Arrhythmia is a significant cardiovascular disease that poses a threat to human health, and its primary diagnosis relies on electrocardiogram (ECG). Implementing computer technology to achieve automatic classification of arrhythmia can effectively avoid human error, improve diagnostic efficiency, and reduce costs. However, most automatic arrhythmia classification algorithms focus on one-dimensional temporal signals, which lack robustness. Therefore, this study proposed an arrhythmia image classification method based on Gramian angular summation field (GASF) and an improved Inception-ResNet-v2 network. Firstly, the data was preprocessed using variational mode decomposition, and data augmentation was performed using a deep convolutional generative adversarial network. Then, GASF was used to transform one-dimensional ECG signals into two-dimensional images, and an improved Inception-ResNet-v2 network was utilized to implement the five arrhythmia classifications recommended by the AAMI (N, V, S, F, and Q). The experimental results on the MIT-BIH Arrhythmia Database showed that the proposed method achieved an overall classification accuracy of 99.52% and 95.48% under the intra-patient and inter-patient paradigms, respectively. The arrhythmia classification performance of the improved Inception-ResNet-v2 network in this study outperforms other methods, providing a new approach for deep learning-based automatic arrhythmia classification.

CT and MRI fusion based on generative adversarial network and convolutional neural networks under image enhancement / 生物医学工程学杂志

Aiming at the problems of missing important features, inconspicuous details and unclear textures in the fusion of multimodal medical images, this paper proposes a method of computed tomography (CT) image and magnetic resonance imaging (MRI) image fusion using generative adversarial network (GAN) and convolutional neural network (CNN) under image enhancement. The generator aimed at high-frequency feature images and used double discriminators to target the fusion images after inverse transform; Then high-frequency feature images were fused by trained GAN model, and low-frequency feature images were fused by CNN pre-training model based on transfer learning. Experimental results showed that, compared with the current advanced fusion algorithm, the proposed method had more abundant texture details and clearer contour edge information in subjective representation. In the evaluation of objective indicators, Q AB/F, information entropy (IE), spatial frequency (SF), structural similarity (SSIM), mutual information (MI) and visual information fidelity for fusion (VIFF) were 2.0%, 6.3%, 7.0%, 5.5%, 9.0% and 3.3% higher than the best test results, respectively. The fused image can be effectively applied to medical diagnosis to further improve the diagnostic efficiency.

Electrocardiogram data recognition algorithm based on variable scale fusion network model / 生物医学工程学杂志

The judgment of the type of arrhythmia is the key to the prevention and diagnosis of early cardiovascular disease. Therefore, electrocardiogram (ECG) analysis has been widely used as an important basis for doctors to diagnose. However, due to the large differences in ECG signal morphology among different patients and the unbalanced distribution of categories, the existing automatic detection algorithms for arrhythmias have certain difficulties in the identification process. This paper designs a variable scale fusion network model for automatic recognition of heart rhythm types. In this study, a variable-scale fusion network model was proposed for automatic identification of heart rhythm types. The improved ECG generation network (EGAN) module was used to solve the imbalance of ECG data, and the ECG signal was reproduced in two dimensions in the form of gray recurrence plot (GRP) and spectrogram. Combined with the branching structure of the model, the automatic classification of variable-length heart beats was realized. The results of the study were verified by the Massachusetts institute of technology and Beth Israel hospital (MIT-BIH) arrhythmia database, which distinguished eight heart rhythm types. The average accuracy rate reached 99.36%, and the sensitivity and specificity were 96.11% and 99.84%, respectively. In conclusion, it is expected that this method can be used for clinical auxiliary diagnosis and smart wearable devices in the future.

Preliminary study on motion artifacts removal of coronary CT angiography using generative adversarial network / 上海交通大学学报（医学版）

Objective: To investigate the ability of generative adversarial network (GAN) to remove motion artifacts in coronary CT angiography (CTA) images. Methods: Subjects who underwent single-cardiac-cycle multi-phase CTA were included and divided into training and test group. The middle segment of the right coronary artery (RCA) was investigated because its motion artifact is the most prominent among all coronary branches. The patch image of the vessel with motion artifacts was extracted, and paired images without artifacts were considered as reference. The GAN model was established according to the training group. In the test group, vessel images were segmented out of the surrounding tissues by using ITK-SNAP software, including the vessel with artifacts, GAN-generated images and reference images. The Dice coefficients of the vessel with artifacts vs reference image (dice1) and GAN-generated images vs reference image (dice2) were cal-culated. By comparing the difference between dice1 and dice2, GAN's ability in removing motion artifacts was evaluated. Results: Ninety subjects were included. Seventy-one (11 000 images) were randomly selected as the training group, and the other 19 (3 006 images) were as the test group. Based on subjects, dice1 and dice2 of the middle segment of RCA were 0.38±0.19 and 0.50±0.23, re-spectively (P=0.006). Based on images, the values of the middle segment of RCA were 0.38±0.20 and 0.51±0.26, respectively (P=0.000). Conclusion: GAN can significantly reduce the motion artifacts of CTA in the middle segment of RCA and has the potential to act as a new method to remove motion artifacts of coronary CTA images.

Cross-modal retrieval method for thyroid ultrasound image and text based on generative adversarial network / 生物医学工程学杂志

Ultrasonic examination is a common method in thyroid examination, and the results are mainly composed of thyroid ultrasound images and text reports. Implementation of cross modal retrieval method of images and text reports can provide great convenience for doctors and patients, but currently there is no retrieval method to correlate thyroid ultrasound images with text reports. This paper proposes a cross-modal method based on the deep learning and improved cross-modal generative adversarial network: ①the weight sharing constraints between the fully connection layers used to construct the public representation space in the original network are changed to cosine similarity constraints, so that the network can better learn the common representation of different modal data; ②the fully connection layer is added before the cross-modal discriminator to merge the full connection layer of image and text in the original network with weight sharing. Semantic regularization is realized on the basis of inheriting the advantages of the original network weight sharing. The experimental results show that the mean average precision of cross modal retrieval method for thyroid ultrasound image and text report in this paper can reach 0.508, which is significantly higher than the traditional cross-modal method, providing a new method for cross-modal retrieval of thyroid ultrasound image and text report.

Super-resolution construction of intravascular ultrasound images using generative adversarial networks / 南方医科大学学报

The low-resolution ultrasound images have poor visual effects. Herein we propose a method for generating clearer intravascular ultrasound images based on super-resolution reconstruction combined with generative adversarial networks. We used the generative adversarial networks to generate the images by a generator and to estimate the authenticity of the images by a discriminator. Specifically, the low-resolution image was passed through the sub-pixel convolution layer -feature channels to generate -feature maps in the same size, followed by realignment of the corresponding pixels in each feature map into × sub-blocks, which corresponded to the sub-block in a high-resolution image; after amplification, an image with a -time resolution was generated. The generative adversarial networks can obtain a clearer image through continuous optimization. We compared the method (SRGAN) with other methods including Bicubic, super-resolution convolutional network (SRCNN) and efficient sub-pixel convolutional network (ESPCN), and the proposed method resulted in obvious improvements in the peak signal-to-noise ratio (PSNR) by 2.369 dB and in structural similarity index by 1.79% to enhance the diagnostic visual effects of intravascular ultrasound images.